{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7765UFHoyGx6"
      },
      "source": [
        "##### Copyright 2020 The TensorFlow Authors."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "KsOkK8O69PyT"
      },
      "outputs": [],
      "source": [
        "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZS8z-_KeywY9"
      },
      "source": [
        "# TFL 레이어로 Keras 모델 만들기"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "r61fkA2i9Y3_"
      },
      "source": [
        "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
        "  <td><a target=\"_blank\" href=\"https://www.tensorflow.org/lattice/tutorials/keras_layers\"><img src=\"https://www.tensorflow.org/images/tf_logo_32px.png\">TensorFlow.org에서 보기</a></td>\n",
        "  <td><a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/docs-l10n/blob/master/site/ko/lattice/tutorials/keras_layers.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\">Google Colab에서 실행하기</a></td>\n",
        "  <td><a target=\"_blank\" href=\"https://github.com/tensorflow/docs-l10n/blob/master/site/ko/lattice/tutorials/keras_layers.ipynb\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\">GitHub에서 소스 보기</a></td>\n",
        "  <td><a href=\"https://storage.googleapis.com/tensorflow_docs/docs-l10n/site/ko/lattice/tutorials/keras_layers.ipynb\"><img src=\"https://www.tensorflow.org/images/download_logo_32px.png\">노트북 다운로드하기</a></td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ecLbJCvJSSCd"
      },
      "source": [
        "## 개요\n",
        "\n",
        "TFL Keras 레이어를 사용하여 단조 및 기타 형상 제약 조건이 있는 Keras 모델을 구성할 수 있습니다. 이 예제에서는 TFL 레이어를 사용하여 UCI heart 데이터세트에 대해 보정된 격자 모델을 구축하고 훈련합니다.\n",
        "\n",
        "보정된 격자 모델에서 각 특성은 `tfl.layers.PWLCalibration` 또는 `tfl.layers.CategoricalCalibration` 레이어에 의해 변환되고 결과는 `tfl.layers.Lattice`를 사용하여 비선형적으로 융합됩니다."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "x769lI12IZXB"
      },
      "source": [
        "## 설정"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fbBVAR6UeRN5"
      },
      "source": [
        "TF Lattice 패키지 설치하기"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bpXjJKpSd3j4"
      },
      "outputs": [],
      "source": [
        "#@test {\"skip\": true}\n",
        "!pip install tensorflow-lattice pydot"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jSVl9SHTeSGX"
      },
      "source": [
        "필수 패키지 가져오기"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "both",
        "id": "pm0LD8iyIZXF"
      },
      "outputs": [],
      "source": [
        "import tensorflow as tf\n",
        "\n",
        "import logging\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "import sys\n",
        "import tensorflow_lattice as tfl\n",
        "from tensorflow import feature_column as fc\n",
        "logging.disable(sys.maxsize)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "svPuM6QNxlrH"
      },
      "source": [
        "UCI Statlog(Heart) 데이터세트 다운로드하기"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "both",
        "id": "PG3pFtK-IZXM"
      },
      "outputs": [],
      "source": [
        "# UCI Statlog (Heart) dataset.\n",
        "csv_file = tf.keras.utils.get_file(\n",
        "    'heart.csv', 'http://storage.googleapis.com/applied-dl/heart.csv')\n",
        "training_data_df = pd.read_csv(csv_file).sample(\n",
        "    frac=1.0, random_state=41).reset_index(drop=True)\n",
        "training_data_df.head()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nKkAw12SxvGG"
      },
      "source": [
        "이 가이드에서 훈련에 사용되는 기본값 설정하기"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "both",
        "id": "krAJBE-yIZXR"
      },
      "outputs": [],
      "source": [
        "LEARNING_RATE = 0.1\n",
        "BATCH_SIZE = 128\n",
        "NUM_EPOCHS = 100"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0TGfzhPHzpix"
      },
      "source": [
        "## 순차형 Keras 모델\n",
        "\n",
        "이 예제는 순차형 Keras 모델을 생성하고 TFL 레이어만 사용합니다.\n",
        "\n",
        "격자 레이어는 `input[i]`이 `[0, lattice_sizes[i] - 1.0]` 내에 있을 것으로 예상하므로 보정 레이어보다 먼저 격자 크기를 정의해야 보정 레이어의 출력 범위를 올바르게 지정할 수 있습니다.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "nOQWqPAbQS3o"
      },
      "outputs": [],
      "source": [
        "# Lattice layer expects input[i] to be within [0, lattice_sizes[i] - 1.0], so\n",
        "lattice_sizes = [3, 2, 2, 2, 2, 2, 2]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "W3DnEKWvQYXm"
      },
      "source": [
        "`tfl.layers.ParallelCombination` 레이어를 사용하여 순차형 모델을 생성하기 위해 병렬로 실행해야 하는 보정 레이어를 그룹화합니다.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "o_hyk5GkQfl8"
      },
      "outputs": [],
      "source": [
        "combined_calibrators = tfl.layers.ParallelCombination()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BPZsSUZiQiwc"
      },
      "source": [
        "각 특성에 대한 보정 레이어를 만들고 병렬 조합 레이어에 추가합니다. 숫자 특성에는 `tfl.layers.PWLCalibration`을 사용하고 범주형 특성에는 `tfl.layers.CategoricalCalibration`을 사용합니다."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "DXPc6rSGxzFZ"
      },
      "outputs": [],
      "source": [
        "# ############### age ###############\n",
        "calibrator = tfl.layers.PWLCalibration(\n",
        "    # Every PWLCalibration layer must have keypoints of piecewise linear\n",
        "    # function specified. Easiest way to specify them is to uniformly cover\n",
        "    # entire input range by using numpy.linspace().\n",
        "    input_keypoints=np.linspace(\n",
        "        training_data_df['age'].min(), training_data_df['age'].max(), num=5),\n",
        "    # You need to ensure that input keypoints have same dtype as layer input.\n",
        "    # You can do it by setting dtype here or by providing keypoints in such\n",
        "    # format which will be converted to deisred tf.dtype by default.\n",
        "    dtype=tf.float32,\n",
        "    # Output range must correspond to expected lattice input range.\n",
        "    output_min=0.0,\n",
        "    output_max=lattice_sizes[0] - 1.0,\n",
        ")\n",
        "combined_calibrators.append(calibrator)\n",
        "\n",
        "# ############### sex ###############\n",
        "# For boolean features simply specify CategoricalCalibration layer with 2\n",
        "# buckets.\n",
        "calibrator = tfl.layers.CategoricalCalibration(\n",
        "    num_buckets=2,\n",
        "    output_min=0.0,\n",
        "    output_max=lattice_sizes[1] - 1.0,\n",
        "    # Initializes all outputs to (output_min + output_max) / 2.0.\n",
        "    kernel_initializer='constant')\n",
        "combined_calibrators.append(calibrator)\n",
        "\n",
        "# ############### cp ###############\n",
        "calibrator = tfl.layers.PWLCalibration(\n",
        "    # Here instead of specifying dtype of layer we convert keypoints into\n",
        "    # np.float32.\n",
        "    input_keypoints=np.linspace(1, 4, num=4, dtype=np.float32),\n",
        "    output_min=0.0,\n",
        "    output_max=lattice_sizes[2] - 1.0,\n",
        "    monotonicity='increasing',\n",
        "    # You can specify TFL regularizers as a tuple ('regularizer name', l1, l2).\n",
        "    kernel_regularizer=('hessian', 0.0, 1e-4))\n",
        "combined_calibrators.append(calibrator)\n",
        "\n",
        "# ############### trestbps ###############\n",
        "calibrator = tfl.layers.PWLCalibration(\n",
        "    # Alternatively, you might want to use quantiles as keypoints instead of\n",
        "    # uniform keypoints\n",
        "    input_keypoints=np.quantile(training_data_df['trestbps'],\n",
        "                                np.linspace(0.0, 1.0, num=5)),\n",
        "    dtype=tf.float32,\n",
        "    # Together with quantile keypoints you might want to initialize piecewise\n",
        "    # linear function to have 'equal_slopes' in order for output of layer\n",
        "    # after initialization to preserve original distribution.\n",
        "    kernel_initializer='equal_slopes',\n",
        "    output_min=0.0,\n",
        "    output_max=lattice_sizes[3] - 1.0,\n",
        "    # You might consider clamping extreme inputs of the calibrator to output\n",
        "    # bounds.\n",
        "    clamp_min=True,\n",
        "    clamp_max=True,\n",
        "    monotonicity='increasing')\n",
        "combined_calibrators.append(calibrator)\n",
        "\n",
        "# ############### chol ###############\n",
        "calibrator = tfl.layers.PWLCalibration(\n",
        "    # Explicit input keypoint initialization.\n",
        "    input_keypoints=[126.0, 210.0, 247.0, 286.0, 564.0],\n",
        "    dtype=tf.float32,\n",
        "    output_min=0.0,\n",
        "    output_max=lattice_sizes[4] - 1.0,\n",
        "    # Monotonicity of calibrator can be decreasing. Note that corresponding\n",
        "    # lattice dimension must have INCREASING monotonicity regardless of\n",
        "    # monotonicity direction of calibrator.\n",
        "    monotonicity='decreasing',\n",
        "    # Convexity together with decreasing monotonicity result in diminishing\n",
        "    # return constraint.\n",
        "    convexity='convex',\n",
        "    # You can specify list of regularizers. You are not limited to TFL\n",
        "    # regularizrs. Feel free to use any :)\n",
        "    kernel_regularizer=[('laplacian', 0.0, 1e-4),\n",
        "                        tf.keras.regularizers.l1_l2(l1=0.001)])\n",
        "combined_calibrators.append(calibrator)\n",
        "\n",
        "# ############### fbs ###############\n",
        "calibrator = tfl.layers.CategoricalCalibration(\n",
        "    num_buckets=2,\n",
        "    output_min=0.0,\n",
        "    output_max=lattice_sizes[5] - 1.0,\n",
        "    # For categorical calibration layer monotonicity is specified for pairs\n",
        "    # of indices of categories. Output for first category in pair will be\n",
        "    # smaller than output for second category.\n",
        "    #\n",
        "    # Don't forget to set monotonicity of corresponding dimension of Lattice\n",
        "    # layer to '1'.\n",
        "    monotonicities=[(0, 1)],\n",
        "    # This initializer is identical to default one('uniform'), but has fixed\n",
        "    # seed in order to simplify experimentation.\n",
        "    kernel_initializer=tf.keras.initializers.RandomUniform(\n",
        "        minval=0.0, maxval=lattice_sizes[5] - 1.0, seed=1))\n",
        "combined_calibrators.append(calibrator)\n",
        "\n",
        "# ############### restecg ###############\n",
        "calibrator = tfl.layers.CategoricalCalibration(\n",
        "    num_buckets=3,\n",
        "    output_min=0.0,\n",
        "    output_max=lattice_sizes[6] - 1.0,\n",
        "    # Categorical monotonicity can be partial order.\n",
        "    monotonicities=[(0, 1), (0, 2)],\n",
        "    # Categorical calibration layer supports standard Keras regularizers.\n",
        "    kernel_regularizer=tf.keras.regularizers.l1_l2(l1=0.001),\n",
        "    kernel_initializer='constant')\n",
        "combined_calibrators.append(calibrator)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "inyNlSBeQyp7"
      },
      "source": [
        "그런 다음 calibrator의 출력을 비선형적으로 융합하기 위해 격자 레이어를 만듭니다.\n",
        "\n",
        "필요한 차원에 대해 증가할 격자의 단조를 지정해야 합니다. 보정에서 단조로운 방향의 구성은 단조의 엔드 투 엔드 방향을 올바르게 만듭니다. 여기에는 CategoricalCalibration 레이어의 부분 단조가 포함됩니다."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "DNCc9oBTRo6w"
      },
      "outputs": [],
      "source": [
        "lattice = tfl.layers.Lattice(\n",
        "    lattice_sizes=lattice_sizes,\n",
        "    monotonicities=[\n",
        "        'increasing', 'none', 'increasing', 'increasing', 'increasing',\n",
        "        'increasing', 'increasing'\n",
        "    ],\n",
        "    output_min=0.0,\n",
        "    output_max=1.0)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "T5q2InayRpDr"
      },
      "source": [
        "그런 다음 결합된 calibrator 및 격자 레이어를 사용하여 순차형 모델을 만들 수 있습니다."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "xX6lroYZQy3L"
      },
      "outputs": [],
      "source": [
        "model = tf.keras.models.Sequential()\n",
        "model.add(combined_calibrators)\n",
        "model.add(lattice)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "W3UFxD3fRzIC"
      },
      "source": [
        "훈련은 다른 Keras 모델과 동일하게 동작합니다."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "2jz4JvI-RzSj"
      },
      "outputs": [],
      "source": [
        "features = training_data_df[[\n",
        "    'age', 'sex', 'cp', 'trestbps', 'chol', 'fbs', 'restecg'\n",
        "]].values.astype(np.float32)\n",
        "target = training_data_df[['target']].values.astype(np.float32)\n",
        "\n",
        "model.compile(\n",
        "    loss=tf.keras.losses.mean_squared_error,\n",
        "    optimizer=tf.keras.optimizers.Adagrad(learning_rate=LEARNING_RATE))\n",
        "model.fit(\n",
        "    features,\n",
        "    target,\n",
        "    batch_size=BATCH_SIZE,\n",
        "    epochs=NUM_EPOCHS,\n",
        "    validation_split=0.2,\n",
        "    shuffle=False,\n",
        "    verbose=0)\n",
        "\n",
        "model.evaluate(features, target)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RTHoW_5lxwT5"
      },
      "source": [
        "## 함수형 Keras 모델\n",
        "\n",
        "이 예제에서는 Keras 모델 생성을 위한 함수형 API를 사용합니다.\n",
        "\n",
        "이전 섹션에서 언급했듯이 격자 레이어는 `input[i]`가 `[0, lattice_sizes[i] - 1.0]` 내에 있을 것으로 예상되므로 보정 레이어보다 먼저 격자 크기를 정의해야 보정 레이어의 출력 범위를 적절하게 지정할 수 있습니다."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "gJjUYvBuW1qE"
      },
      "outputs": [],
      "source": [
        "# We are going to have 2-d embedding as one of lattice inputs.\n",
        "lattice_sizes = [3, 2, 2, 3, 3, 2, 2]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Z03qY5MYW1yT"
      },
      "source": [
        "각 특성에 대해 입력 레이어와 보정 레이어를 만들어야 합니다. 숫자 특성에는 `tfl.layers.PWLCalibration`을 사용하고 범주형 특성에는 `tfl.layers.CategoricalCalibration`을 사용합니다."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "DCIUz8apzs0l"
      },
      "outputs": [],
      "source": [
        "model_inputs = []\n",
        "lattice_inputs = []\n",
        "# ############### age ###############\n",
        "age_input = tf.keras.layers.Input(shape=[1], name='age')\n",
        "model_inputs.append(age_input)\n",
        "age_calibrator = tfl.layers.PWLCalibration(\n",
        "    # Every PWLCalibration layer must have keypoints of piecewise linear\n",
        "    # function specified. Easiest way to specify them is to uniformly cover\n",
        "    # entire input range by using numpy.linspace().\n",
        "    input_keypoints=np.linspace(\n",
        "        training_data_df['age'].min(), training_data_df['age'].max(), num=5),\n",
        "    # You need to ensure that input keypoints have same dtype as layer input.\n",
        "    # You can do it by setting dtype here or by providing keypoints in such\n",
        "    # format which will be converted to deisred tf.dtype by default.\n",
        "    dtype=tf.float32,\n",
        "    # Output range must correspond to expected lattice input range.\n",
        "    output_min=0.0,\n",
        "    output_max=lattice_sizes[0] - 1.0,\n",
        "    monotonicity='increasing',\n",
        "    name='age_calib',\n",
        ")(\n",
        "    age_input)\n",
        "lattice_inputs.append(age_calibrator)\n",
        "\n",
        "# ############### sex ###############\n",
        "# For boolean features simply specify CategoricalCalibration layer with 2\n",
        "# buckets.\n",
        "sex_input = tf.keras.layers.Input(shape=[1], name='sex')\n",
        "model_inputs.append(sex_input)\n",
        "sex_calibrator = tfl.layers.CategoricalCalibration(\n",
        "    num_buckets=2,\n",
        "    output_min=0.0,\n",
        "    output_max=lattice_sizes[1] - 1.0,\n",
        "    # Initializes all outputs to (output_min + output_max) / 2.0.\n",
        "    kernel_initializer='constant',\n",
        "    name='sex_calib',\n",
        ")(\n",
        "    sex_input)\n",
        "lattice_inputs.append(sex_calibrator)\n",
        "\n",
        "# ############### cp ###############\n",
        "cp_input = tf.keras.layers.Input(shape=[1], name='cp')\n",
        "model_inputs.append(cp_input)\n",
        "cp_calibrator = tfl.layers.PWLCalibration(\n",
        "    # Here instead of specifying dtype of layer we convert keypoints into\n",
        "    # np.float32.\n",
        "    input_keypoints=np.linspace(1, 4, num=4, dtype=np.float32),\n",
        "    output_min=0.0,\n",
        "    output_max=lattice_sizes[2] - 1.0,\n",
        "    monotonicity='increasing',\n",
        "    # You can specify TFL regularizers as tuple ('regularizer name', l1, l2).\n",
        "    kernel_regularizer=('hessian', 0.0, 1e-4),\n",
        "    name='cp_calib',\n",
        ")(\n",
        "    cp_input)\n",
        "lattice_inputs.append(cp_calibrator)\n",
        "\n",
        "# ############### trestbps ###############\n",
        "trestbps_input = tf.keras.layers.Input(shape=[1], name='trestbps')\n",
        "model_inputs.append(trestbps_input)\n",
        "trestbps_calibrator = tfl.layers.PWLCalibration(\n",
        "    # Alternatively, you might want to use quantiles as keypoints instead of\n",
        "    # uniform keypoints\n",
        "    input_keypoints=np.quantile(training_data_df['trestbps'],\n",
        "                                np.linspace(0.0, 1.0, num=5)),\n",
        "    dtype=tf.float32,\n",
        "    # Together with quantile keypoints you might want to initialize piecewise\n",
        "    # linear function to have 'equal_slopes' in order for output of layer\n",
        "    # after initialization to preserve original distribution.\n",
        "    kernel_initializer='equal_slopes',\n",
        "    output_min=0.0,\n",
        "    output_max=lattice_sizes[3] - 1.0,\n",
        "    # You might consider clamping extreme inputs of the calibrator to output\n",
        "    # bounds.\n",
        "    clamp_min=True,\n",
        "    clamp_max=True,\n",
        "    monotonicity='increasing',\n",
        "    name='trestbps_calib',\n",
        ")(\n",
        "    trestbps_input)\n",
        "lattice_inputs.append(trestbps_calibrator)\n",
        "\n",
        "# ############### chol ###############\n",
        "chol_input = tf.keras.layers.Input(shape=[1], name='chol')\n",
        "model_inputs.append(chol_input)\n",
        "chol_calibrator = tfl.layers.PWLCalibration(\n",
        "    # Explicit input keypoint initialization.\n",
        "    input_keypoints=[126.0, 210.0, 247.0, 286.0, 564.0],\n",
        "    output_min=0.0,\n",
        "    output_max=lattice_sizes[4] - 1.0,\n",
        "    # Monotonicity of calibrator can be decreasing. Note that corresponding\n",
        "    # lattice dimension must have INCREASING monotonicity regardless of\n",
        "    # monotonicity direction of calibrator.\n",
        "    monotonicity='decreasing',\n",
        "    # Convexity together with decreasing monotonicity result in diminishing\n",
        "    # return constraint.\n",
        "    convexity='convex',\n",
        "    # You can specify list of regularizers. You are not limited to TFL\n",
        "    # regularizrs. Feel free to use any :)\n",
        "    kernel_regularizer=[('laplacian', 0.0, 1e-4),\n",
        "                        tf.keras.regularizers.l1_l2(l1=0.001)],\n",
        "    name='chol_calib',\n",
        ")(\n",
        "    chol_input)\n",
        "lattice_inputs.append(chol_calibrator)\n",
        "\n",
        "# ############### fbs ###############\n",
        "fbs_input = tf.keras.layers.Input(shape=[1], name='fbs')\n",
        "model_inputs.append(fbs_input)\n",
        "fbs_calibrator = tfl.layers.CategoricalCalibration(\n",
        "    num_buckets=2,\n",
        "    output_min=0.0,\n",
        "    output_max=lattice_sizes[5] - 1.0,\n",
        "    # For categorical calibration layer monotonicity is specified for pairs\n",
        "    # of indices of categories. Output for first category in pair will be\n",
        "    # smaller than output for second category.\n",
        "    #\n",
        "    # Don't forget to set monotonicity of corresponding dimension of Lattice\n",
        "    # layer to '1'.\n",
        "    monotonicities=[(0, 1)],\n",
        "    # This initializer is identical to default one ('uniform'), but has fixed\n",
        "    # seed in order to simplify experimentation.\n",
        "    kernel_initializer=tf.keras.initializers.RandomUniform(\n",
        "        minval=0.0, maxval=lattice_sizes[5] - 1.0, seed=1),\n",
        "    name='fbs_calib',\n",
        ")(\n",
        "    fbs_input)\n",
        "lattice_inputs.append(fbs_calibrator)\n",
        "\n",
        "# ############### restecg ###############\n",
        "restecg_input = tf.keras.layers.Input(shape=[1], name='restecg')\n",
        "model_inputs.append(restecg_input)\n",
        "restecg_calibrator = tfl.layers.CategoricalCalibration(\n",
        "    num_buckets=3,\n",
        "    output_min=0.0,\n",
        "    output_max=lattice_sizes[6] - 1.0,\n",
        "    # Categorical monotonicity can be partial order.\n",
        "    monotonicities=[(0, 1), (0, 2)],\n",
        "    # Categorical calibration layer supports standard Keras regularizers.\n",
        "    kernel_regularizer=tf.keras.regularizers.l1_l2(l1=0.001),\n",
        "    kernel_initializer='constant',\n",
        "    name='restecg_calib',\n",
        ")(\n",
        "    restecg_input)\n",
        "lattice_inputs.append(restecg_calibrator)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Fr0k8La_YgQG"
      },
      "source": [
        "그런 다음 calibrator의 출력을 비선형적으로 융합하기 위해 격자 레이어를 만듭니다.\n",
        "\n",
        "필요한 차원에 대해 증가할 격자의 단조를 지정해야 합니다. 보정에서 단조로운 방향의 구성은 단조의 엔드 투 엔드 방향을 올바르게 만듭니다. 여기에는 `tfl.layers.CategoricalCalibration` 레이어의 부분 단조가 포함됩니다."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "X15RE0NybNbU"
      },
      "outputs": [],
      "source": [
        "lattice = tfl.layers.Lattice(\n",
        "    lattice_sizes=lattice_sizes,\n",
        "    monotonicities=[\n",
        "        'increasing', 'none', 'increasing', 'increasing', 'increasing',\n",
        "        'increasing', 'increasing'\n",
        "    ],\n",
        "    output_min=0.0,\n",
        "    output_max=1.0,\n",
        "    name='lattice',\n",
        ")(\n",
        "    lattice_inputs)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "31VzsnMCA9dh"
      },
      "source": [
        "모델에 더 많은 유연성을 추가하기 위해 출력 보정 레이어를 추가합니다."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "efCP3Yx2A9n7"
      },
      "outputs": [],
      "source": [
        "model_output = tfl.layers.PWLCalibration(\n",
        "    input_keypoints=np.linspace(0.0, 1.0, 5),\n",
        "    name='output_calib',\n",
        ")(\n",
        "    lattice)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1SURnNl8bNgw"
      },
      "source": [
        "이제 입력과 출력을 사용하여 모델을 만들 수 있습니다."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "7gY-VXuYbZLa"
      },
      "outputs": [],
      "source": [
        "model = tf.keras.models.Model(\n",
        "    inputs=model_inputs,\n",
        "    outputs=model_output)\n",
        "tf.keras.utils.plot_model(model, rankdir='LR')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tvFJTs94bZXK"
      },
      "source": [
        "훈련은 다른 Keras 모델과 동일하게 동작합니다. 설정에서 입력 특성은 별도의 텐서로 전달됩니다."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "vMQTGbFAYgYS"
      },
      "outputs": [],
      "source": [
        "feature_names = ['age', 'sex', 'cp', 'trestbps', 'chol', 'fbs', 'restecg']\n",
        "features = np.split(\n",
        "    training_data_df[feature_names].values.astype(np.float32),\n",
        "    indices_or_sections=len(feature_names),\n",
        "    axis=1)\n",
        "target = training_data_df[['target']].values.astype(np.float32)\n",
        "\n",
        "model.compile(\n",
        "    loss=tf.keras.losses.mean_squared_error,\n",
        "    optimizer=tf.keras.optimizers.Adagrad(LEARNING_RATE))\n",
        "model.fit(\n",
        "    features,\n",
        "    target,\n",
        "    batch_size=BATCH_SIZE,\n",
        "    epochs=NUM_EPOCHS,\n",
        "    validation_split=0.2,\n",
        "    shuffle=False,\n",
        "    verbose=0)\n",
        "\n",
        "model.evaluate(features, target)"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [],
      "name": "keras_layers.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
